Method, apparatus and system for biometric identification

ABSTRACT

Method and apparatus for processing a biometric measurement signal using a computing device, including receiving a biometric measurement signal generated by contact with a single individual, extracting at least one periodic fragment from the biometric measurement signal, generating first feature data at least partially based on the at least one extracted periodic fragment, determining second feature data from the first feature data by removing data from the first feature data using robust principal component analysis, determining whether a match exists between the second feature data and defined biometric data associated with a known individual by processing the second feature data and the defined biometric data using a machine learning technique, and in response to determining a match exists between the second feature data and the defined biometric data, transmitting a signal indicating that the single individual is the known individual.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part of U.S. patent applicationSer. No. 14/869,088 titled “Multi-Modal Biometric Identification,” filedSep. 29, 2015, the contents of which is incorporated herein by referencein its entirety.

TECHNICAL FIELD

This disclosure relates in general to identify a unique individual witha biometric measurement signal using machine learning techniques such asdeep learning.

BACKGROUND

Biometric information has been widely used in identification ofindividuals in various scenarios, such as access control. Suchinformation conventionally includes fingerprints, DNA, eye retinas,facial characteristics, and so forth.

Deep learning (DL) is a branch of machine learning techniques formodeling high-level abstractions in data. DL techniques have been usedfor applications such as face recognition, voice-to-text conversion, andfor assisting medical diagnostics, e.g. disease classification usingrecorded physiological signals. DL techniques can use, for example,multiple processing layers with complex structure, or multiplenon-linear transformations.

SUMMARY

Disclosed herein are implementations of methods, apparatuses and systemsfor biometric identification.

In one aspect, the present disclosure includes a method for processing abiometric measurement signal using a computing device comprisingreceiving a biometric measurement signal generated by contact with asingle individual, extracting periodic fragments from the biometricmeasurement signal, generating first feature data based on the extractedperiodic fragments from the biometric measurement signal, determiningsecond feature data from the first feature data by removing noisy datafrom the first feature data using robust principal component analysis(RPCA), determining whether a match exists between the second featuredata and defined biometric data associated with a known individual byprocessing the second feature data and the defined biometric data usinga machine learning technique, and in response to determining a matchexists between the second feature data and the defined biometric data,transmitting a signal indicating that the single individual is the knownindividual.

In another aspect, the present disclosure includes an apparatus forprocessing a biometric measurement signal comprising a non-transitorymemory and a processor configured to execute instructions stored in thenon-transitory memory to receive a biometric measurement signalgenerated by contact with a single individual, extract periodicfragments from the biometric measurement signal, generate first featuredata based on the extracted periodic fragments, determine second featuredata from the first feature data by removing noisy data from the firstfeature data using RPCA, determine whether a match exists between thesecond feature data and defined biometric data associated with a knownindividual by processing the second feature data and the definedbiometric data using a machine learning technique, and in response todetermining a match exists between the second feature data and thedefined biometric data, transmit a signal indicating that the singleindividual is the known individual.

In another aspect, the present disclosure includes an apparatuscomprising a body, a biometric sensor coupled to the body to produce abiometric measurement signal when activated by contact with an singleindividual, a memory, and at least one communication device coupled tothe body and controlled by a processor to wirelessly transmit thebiometric measurement signal from the biometric sensor to an externalserver, wherein first feature data is generated based on periodicfragments extracted from the biometric measurement signal, wirelesslyreceive a first signal from the external server indicative of abiometric identification data generated from the biometric measurementsignal, wherein the first signal is determined based on second featuredata generated by removing noisy data from the first feature data usingRPCA, and the second feature data is compared with defined biometricdata associated with a known individual using a machine learningtechnique to determine whether a match exists indicating that the singleindividual is the known individual, and wirelessly transmit a secondsignal to an identification device in response to a match indicatingthat the single individual is the known individual.

The embodiments or implementations can be configured as executablecomputer program instructions stored in computer storages such asmemory.

BRIEF DESCRIPTION OF THE DRAWINGS

The description here makes reference to the accompanying drawingswherein like reference numerals refer to like parts throughout theseveral views, and wherein:

FIG. 1 is a diagram of a system configuration for a biometricidentification device and a computer device according to implementationsof this disclosure;

FIG. 2 is a block diagram of a hardware configuration for a biometricidentification device and/or a computing device according toimplementations of this disclosure;

FIG. 3 is a block diagram of a hardware configuration for a biometricidentification device and/or a computing device according toimplementations of this disclosure;

FIG. 4 is a diagram of an example biometric identification deviceaccording to implementations of this disclosure;

FIG. 5 is a diagram of an example biometric identification deviceaccording to implementations of this disclosure;

FIG. 6 is a flowchart showing an example process of processing abiometric measurement signal according to implementations of thisdisclosure;

FIG. 7 is a flowchart showing an example process of comparing biometricmeasurement data of a user and a known individual according toimplementations of this disclosure;

FIG. 8 is a flowchart showing an example setup process using aconvolutional neural network (CNN) model according to implementations ofthis disclosure;

FIG. 9 is a flowchart showing an example process of determining whethera match exists using the CNN model in FIG. 8 according to implementationof this disclosure; and

FIG. 10 is a graph showing example feature data extracted from an ECGsignal, and decomposition of the feature data as a result of usingrobust principal component analysis (RPCA) according to implementationsof this disclosure.

DETAILED DESCRIPTION

Example implementations of the present disclosure will be describedbelow with reference to the accompanying drawings. The same numbersacross the drawings set forth in the following description represent thesame or similar elements, unless differently expressed. Theimplementations set forth in the following description do not representall implementations or embodiments consistent with the presentdisclosure; on the contrary, they are only examples of apparatuses andmethods in accordance with some aspects of this disclosure as detailedin the claims.

A biometric characteristic is applicable as a means to identify aperson, if for any individual the characteristic is: universal, meaningthat any individual possesses one; easily measured, both technically andprocedurally; unique, meaning that no two individuals share identicalmeasurements; and permanent, meaning that the characteristic of theindividual does not change over time. Further, when implemented by awearable device, the biometric measurement signal can have instantaneitythat means the measurement should take a small amount of time; accuracythat means the measurement should not misidentify the individual asanother person; and security that means the characteristic should not beeasily copied or inappropriately possessed by other individuals. Thebiometric measurement signal used in implementations of this disclosurecan be associated with, for example, a voice, a hand-writing, a handgeometry, a fingerprint, a palm print, an iris characteristic, a facialcharacteristic, an electrocardiogram (ECG) signal, anelectroencephalography (EEG) signal, a photoplethysmography (PPG)signal, an electromyography (EMG) signal, or a combination of the above.

FIG. 1 is a diagram of a system configuration 100 for a biometricidentification device 110 and a computing device 112 according toimplementations of this disclosure.

Biometric identification device 110 as shown is a wearable biometricidentification device, namely a device worn around an individual'swrist. However, other devices can be used. For example, device 110 couldinstead be implemented by another wearable device such as a ring ornecklace. Alternatively, device 110 could be implemented as anotherportable device that is configured to travel with an individual, but notbe worn by the individual, such as a device similar in form to a keyfob. Computing device 112 can be implemented by any configuration of oneor more computers, such as a remote server computer, a personalcomputer, a laptop computer, a tablet computer, a cell phone, a personaldata assistant (PDA), or a computing service provided by a computingservice provider, e.g., a website, cloud computing. For example, certainoperations described herein can be performed by a computer (e.g., aserver computer) in the form of multiple groups of computers that are atdifferent geographic locations and can or can not communicate with oneanother, such as by way of network 120. While certain operations can beshared by multiple computers, in some implementations, differentcomputers are assigned different operations. For example, one or morecomputing devices, such as a cell phone, could be used to receive andprocess biometric identification data as described hereinafter, andtransmit a signal to biometric identification device 110 and/orelsewhere confirming or denying a match.

In another implementation, an intermediate device (not shown in FIG. 1),can be used to establish a connection to biometric identification device110 carried or wore by a user, receive a biometric measurement signal ofthe user from device 110, transmit the biometric measurement signal tocomputing device 112, such as a remote server, to process the biometricmeasurement signal. The computing device determines whether the user isa known individual, and transmits a result signal back to theintermediate device. Prior to transmitting the biometric measurementsignal to computing device 112, the intermediate device can pre-processthe signal. The result signal transmitted back from computing device 112can be used to indicate the user at the intermediate device, oralternatively, the result signal is further transmitted to biometricidentification device 110 by the intermediate device and indicate theuser at device 110. The intermediate device can be a computer or part ofa computer, e.g., a cell phone, a PDA, a tablet computer, or a personalcomputer.

In some implementations, one or more computing devices 112, such as acell phone, can receive a result signal from a remote device (not shownin FIG. 1), after an identity of the user carrying or wearing biometricidentification device 110 is determined. The remote device itself canalso be a computer or part of a computer, e.g., another cell phone or aremote server.

Network 150 can be one or more communications networks of any suitabletype in any combination, including networks using Bluetoothcommunications, infrared communications, near field connections (NFC),wireless networks, wired networks, local area networks (LAN), wide areanetworks (WAN), cellular data networks and the Internet. Biometricidentification device 110 and computing device 112 can communicate witheach other via network 120. In the implementations described herein, onenetwork 150 is shown. Where more than one computing device 112 is used.In some implementations, each computing device 112 can be connected tothe same network or to different networks.

FIG. 2 is a block diagram of a hardware configuration for a biometricidentification device and/or a computing device according toimplementations of this disclosure. For example, biometricidentification device 110 and/or computing device 112 can use hardwareconfiguration 200.

Hardware configuration 200 can include at least one processor such ascentral processing unit (CPU) 210. Alternatively, CPU 210 can be anyother type of device, or multiple devices, capable of manipulating orprocessing information now-existing or hereafter developed. Although theexamples herein can be practiced with a single processor as shown,advantages in speed and efficiency can be achieved using more than oneprocessor.

Memory 220, such as a random access memory device (RAM), a read-onlymemory device (ROM), or any other suitable type of storage device,stores code and data that can be accessed by CPU 210 using a bus 230.The code can include an operating system and one or more applicationprograms processing and/or outputting the data. As will be discussed indetail below, an application program can include software components inthe form of computer executable program instructions that cause CPU 210to perform some or all of the operations and methods described herein.In some implementations, hardware configuration 200 is used to implementcomputing device 112, in which an application program stored by memory220 can implement some or all of a process according to FIG. 6 asdescribed in more detail below.

Hardware configuration 200 can optionally include a storage device 240in the form of any suitable non-transitory computer readable medium,such as a hard disc drive, a memory device, a flash drive or an opticaldrive. Storage device 240, when present, can provide additional memorywhen high processing requirements exist. Storage device 240 can alsostore any form of data, relating or not relating to biometricidentification.

Hardware configuration 200 can include one or more input devices 250,such as a keyboard, a numerical keypad, a mouse, a microphone, a touchscreen, a sensor, or a gesture-sensitive input device. Through inputdevice 250, data can be input from the user or another device. Forexample, a gesture-sensitive input device can receive different gesturesto switch between different display modes (e.g., heart rate, time, ECG).Any other type of input device 250, including an input device notrequiring user intervention, is possible. For example, input device 250can be a communication device such as a wireless receiver operatingaccording to any wireless protocol for receiving signals. In someimplementations, when hardware configuration 200 is used to implementcomputing device 112, input device 250 can be a wireless receiver forreceiving input signals from biometric identification device 110. Inanother implementation, when hardware configuration 200 is used toimplement biometric identification device 110, input device 250 can awireless receiver for receiving result signals from computing device112. Input device 250 can output signals or data, indicative of theinputs, to CPU 210, e.g., along bus 230.

Hardware configuration 200 can include one or more output devices 260.Output device 260 can be any device transmitting a visual, acoustic, ortactile signal to the user, such as a display, a touch screen, aspeaker, an earphone, a light-emitting diode (LED) indicator, or avibration motor. If output device is a display, for example, it can be aliquid crystal display (LCD), a cathode-ray tube (CRT), or any otheroutput device capable of providing visible output to an individual. Insome cases, an output device 260 can also function as an input device250—a touch screen display configured to receive touch-based input, forexample. Output device 260 can alternatively or additionally be formedof a communication device for transmitting signals. For example, outputdevice 260 can include a wireless transmitter using a protocolcompatible with a wireless receiver of biometric identification device110 to transmit signals from computing device 112 to biometricidentification device 110.

Although FIG. 2 depicts one hardware configuration 200 that canimplement computer device 112, other configurations can be utilized. Theoperations of CPU 210 can be distributed across multiple machines ordevices (each machine or device having one or more of processors) thatcan be coupled directly or across a local area or other network.Memories 220 can be distributed across multiple machines or devices suchas network-based memory or memory in multiple machines performingoperations that can be described herein as being performed using asingle computer or computing device for ease of explanation. Although asingle bus 230 is depicted, multiple buses can be utilized. Further,storage device 240 can be a component of hardware configuration 200 orcan be a shared device that is accessed via a network. The hardwareconfiguration of a computing system as depicted in an example in FIG. 2thus can be implemented in a wide variety of configurations.

As a generalized configuration is represented by hardware configuration200, a more specific hardware configuration for biometric identificationdevice 110 is illustrated as a block diagram in FIG. 3 according toimplementations of this disclosure.

In some implementations, biometric identification device 110 comprisesCPU 210, memory 220, biometric sensor 330, and communication device 340.CPU 210 and memory 220 can be any implementation as set forth in thedescription of FIG. 2. Biometric sensor 330 can be configured to be oneor more biometric sensors that measure one or more biometric measurementsignals of the user or devices that collect biometric data of the user,e.g., by contacting or interacting with the user. The biometricmeasurement can be a process with or without the user providingindication of starting the measurement to biometric identificationdevice 110 and/or inputting data during the measurement process, inwhich the indication or data, if incurred, is transmitted viacommunication device 340. Biometric sensor 330 can be a microphone, acamera, a touch screen, a fingerprint reader, an iris scanner, an ECGsensor, an EEG sensor, a PPG sensor, an EMG sensor, or a combination ofa plurality of the abovementioned sensors.

Communication device 340 is configured to input and/or output signal tobiometric identification device 110, which can be any implementation ofinput device 250 and/or output device 260 or a combination thereof. Insome implementations, communication device 340 further includes adisplay for presenting output to indicate the successful identificationof the user. In a further implementation, the display is a touch screendisplay configured to receive touch-based input, for example, inmanipulating data outputted thereto. In another implementation,communication device 340 can be configured to receive a signal fromcomputing device 112, an intermediate device, or a remote device as setforth in aforementioned description.

A configuration of biometric identification device 110 is described inmore detail with reference to FIG. 4 and FIG. 5.

FIG. 4 is diagram of an example biometric identification deviceaccording to implementations of this disclosure. In this example, device110 is shown as a wristband device 400. Although wristband device 400 isshown as having a module 420 secured to a wrist band 440, other devicesthat can be worn on an individual's body can be used, such as wearableson the user's arms, wrists or fingers. The module 420 of wristbanddevice 400 can include, for example, CPU 210, memory 220, one or morebiometric sensors 330 and one or more communication devices 340.Securing mechanism 460 can also be included to secure band 440 to theuser.

FIG. 5 is diagram of an example biometric identification deviceaccording to implementations of this disclosure. In this example, device110 is shown as a “smart” watch 500.

Although watch 500 is shown as having a module 520 secured to a wristband 540, other devices that can be worn on an individual's body can beused, such as wearables on the user's arms, wrists or fingers. Themodule 520 of watch 500 can include, for example, CPU 210, memory 220,one or more biometric sensors 330 and one or more communication devices340. Securing mechanism 560 can also be included to secure band 540 tothe user.

In some implementations, securing mechanism 460 and/or 560 is a slot andpeg configuration. In other implementations, securing mechanism 460and/or 560 can include a snap-lock configuration. It will be apparent toone skilled in the art in view of the present disclosure that variousconfigurations can be contemplated for securing mechanism 460 and/or560.

FIG. 6 is a flowchart showing an example process of processing abiometric measurement signal according to implementations of thisdisclosure. The operations described in method 600 can be performed atone or more computing devices, e.g., a computing device 112 such as aremote server, or an intermediate device such as a cell phone, or at thebiometric identification device 110, or a combination of the above. Whenan operation is performed by one or more such computing devices, it iscompleted when it is performed by one such computing device. Theoperations described in method 600 can be implemented in memory 220including program instructions executable by CPU 210 that, whenexecuted, cause CPU 210 to perform the operations.

For simplicity of explanation, method 600 is described below asperformed by computing device 112. Accordingly, a biometric measurementsignal can be received at computing device 112 from biometricidentification device 110 through, for example, communication device 340at operation 602. Also in this example, only ECG signals are discussed.However, other types or multiple biometric measurement signals can besimilarly processed. In some implementations, measuring biometricmeasurement signals ends after a defined period of time lapses. In otherimplementations, measuring biometric measurement signals ends whencontact of the individual with a sensor ends.

At operation 602, a biometric measurement signal, generated by biometricidentification device 110 by contact with a user, is received. Thebiometric measurement signal can be of any signal measured and generatedby any kind of biometric sensor set forth herein. The biometricmeasurement signal contains characteristics, information or data that isassociated with the user. In some implementations, the biometricmeasurement signal is an ECG signal. Receiving is defined herein asinputting, acquiring, reading, accessing or in any manner inputting abiometric measurement signal. For example, the biometric measurementsignal is received via communication device 340 of computing device 112,such as a server or a personal computer. In another implementation, thebiometric measurement signal is received via communication device 340 ofan intermediate device, such as a cell phone or a tablet computer, whichfurther transmits the signal to computing device 112. In anotherimplementation, the intermediate device can pre-process the biometricmeasurement signal prior to transmitting the same to computing device112. In another implementation, the pre-processing of the biometricmeasurement signal can be performed at computing device 112.

The pre-processing includes a number of manipulations to the biometricmeasurement signal to ensure data integrity and to prepare the signalfor subsequent processing. The type of pre-processing varies accordingto type of signal, but it generally involves removing noise of rawsignals (e.g., denoising) measured by the sensors. In someimplementations, the pre-processing can include removing baseline wanderin the raw signals. This processing generally adjusts the input signalsduring one measurement cycle to a common baseline. In anotherimplementation, the pre-processing can include filtering, such as usinga band pass filter, which can remove any undesirable data shifts thatoccurred while the raw signals were being measured and to reduce thepresence of data outside of a range to be observed (e.g., outliers). Insome implementations, techniques such as wavelet transform and supportvector machines (SVM) can used to remove outliers and noises from thedata.

At operation 604, one or more periodic fragment is extracted from thebiometric measurement signal. The one or more periodic fragments can beportions of the biometric signal that presents periodicalcharacteristics. Optionally, the biometric measurement signal can bepre-processed. In some implementations, the biometric measurement signalcan be an ECG signal and the one or more periodic fragments are PQRSTcycles of the ECG signal, in which each periodic fragment contains oneor more complete PQRST cycles. A PQRST cycle of an ECG signal is definedas a portion of the ECG signal that represents one complete heartbeat,which consists of a P-wave, a QRS complex and a T-wave connected intemporal order. The peak of a QRS complex is defined as an R-peak. Forexample, after detecting R-peaks, ECG periodic fragments (e.g., QRScomplexes) can be extracted from the pre-processed signal based onR-peaks by, for example, directly taking signal data around the R-peaks,or any other technique that can be used to extract ECG periodicfragments. In some implementations, an extracted periodic fragment canbe represented by an image, or a vector. Other data structures are alsopossible.

At operation 606, first feature data is generated by computer device112, at least partially based on the extracted periodic fragments fromthe biometric measurement signal. In some implementations, the firstfeature data can be an aggregated data structure, which is generated byaggregating one or more extracted periodic fragments represented asimages or vectors. For example, the periodic fragments can berepresented as images in which each image can represent one or moreperiodic fragment, and an aggregated image can be generated by stackingan image on top of another image.

In another example, the periodic fragments can be represented as vectorsin which each vector can represent one or more periodic fragment, and anaggregated image or matrix can be generated by placing a vector side byside with another vector. The vectors representing periodic fragments,if used, can be either row vectors or column vectors, both beingequivalent to each other. For convenience, the vectors representingperiodic fragments set forth hereinafter refer to row vectors, in whichcase each row of the aggregated matrix represents a periodic fragmentaccordingly. The form and the generation method of the first featuredata are not limited to what is set forth herein. Any person skilled inthe art should realize that other techniques, processes, methods oralgorithms can be used to generate the aggregated data structure thatincorporates characteristic information of the extracted periodicfragments and can be processed by a computer program.

At operation 608, second feature data is determined from the firstfeature data by computer device 112, by removing noisy data from thefirst feature data using robust principal component analysis (RPCA).RPCA is a statistical procedure that can be used in processing corrupteddata, such as noisy images, to decompose the corrupted data into twogeneral components, one representing valid/characteristic components(e.g., valid data) of the data and the other representing noises and/oroutliers (e.g., error data) of the data. For example, when the firstfeature data is an aggregated matrix or image, RPCA is applied todecompose it into two matrices or images, one representing valid dataand the other representing error data. The use of the second featuredata by RPCA effectively reduces noises and outliers from the firstfeature data.

In some implementations, the second feature data can be part or whole ofthe valid data of the first feature data processed by RPCA, and the dataremoved from the first feature data using RPCA can be error data of thefirst feature data. In the addition, the data removed can include, ifany, the discarded part of the valid data of the first feature data(e.g., outliers).

In some implementations, an ordering operation can be further performedon the valid data determined from the first feature data using RPCA, andthe second feature data can be the top portion of the valid data (afirst set of principle components), after the ordering operation hasbeen performed on the valid data.

In an example, the ordering operation can reduce the matrix representingvalid data (e.g., a valid signal matrix) into a row echelon form, andthe top portion can be the top K rows of the row echelon form, in whichK is an integer not greater than the rank of the signal matrix. Forexample, for a valid signal matrix including representation of N ECGperiodic fragments after RPCA reduction, the selection of top K rows cangenerate, as a result, a new K*N feature matrix or image. K and N can beany integer numbers, and the ordering operation and the selection of thetop portion are not limited to the examples set forth in the abovedescription.

For example, FIG. 10 is a graph showing example feature data extractedfrom an ECG signal, and decomposition of the feature data as a result ofusing robust principal component analysis (RPCA) according toimplementations of this disclosure. In FIG. 10, the top image representsan aggregated matrix as the first feature data, which can be anaggregated image of a plurality of periodic fragments extracted from anECG signal measured by biometric identification device 110 for a user.In some implementations, RPCA followed by an ordering operation can beapplied to the top image to decompose it into an image representingvalid data (e.g., a valid image) and another image representing noisesand outliers (e.g., an error image). The middle image of FIG. 10 is anexample valid image. The bottom image of either FIG. 10 is an exampleerror image that shows noises and outliers of the aggregated image afterapplying RPCA.

In some implementations, the representations of the extracted periodicfragments can be in other forms. For example, the second feature datacan be determined from the first feature data using RPCA, and theresults can be converted to visual representations for illustrationpurposes.

At operation 610, whether a match exists between the second feature dataand defined biometric data associated with a known individual isdetermined by processing the second feature data and the definedbiometric data using an image processing technique. The image processingtechnique can include, for example, a machine learning technique, suchas a deep learning (DL) technique, as will be described below. If thematch exists, process 600 proceeds to operation 612; otherwise, process600 can go back to operation 602. In some implementations, before thedetermination, the defined biometric data associated with the knownindividual is generated in a training process by the following steps:firstly, data based on periodic fragments associated with the knownindividual and extracted during a setup process is received; secondly,the defined biometric data (e.g., template data) is generated from thedata based on periodic fragments associated with the known individualusing a same technique used to determine the second feature data.

In some implementations, the defined biometric data can be the secondfeature data associated with the known individual, and the data based onperiodic fragments associated with the known individual can be the firstfeature data associated with the known individual. The data can beextracted using the same technique used to generate the first featuredata set forth in the description of operation 606, and the definedbiometric data can be generated using the same technique used togenerate the second feature data set forth in the description ofoperation 608. The setup process is implemented to define an associationformed between the known individual and the defined biometric data,after which the association is used as the basis of determining a matchbetween the known individual and an individual being identified (e.g., acurrent user in contact with device 110).

In some implementations, the image processing technique comprises a deeplearning (DL) technique. The implemented DL technique, as eithersupervised or unsupervised, can work as a DL model taking one or moreinputs and providing one or more outputs. Further, the DL model can havea capability of classifying images, which means outputting a text,numerical or symbolic label for an input image. Output label from alimited group that is pre-determined can be selected before a trainingprocess for the DL model. Various DL model can be used as imageclassifiers, such as convolutional neural networks (CNN), deep beliefnetworks, stacked denoising auto-encoders, or deep Boltzmann machines.However, implementations of image processing techniques are not limitedto DL techniques. Other techniques for image processing can be used inplace of, or in combination with DL techniques.

In some implementations, the DL technique used can include a CNN model.A CNN model is a DL model including nodes (e.g., artificial neurons)structured in a plurality of interconnected layers. A node can be usedfor outputting a value based on a plurality of input values; thefunction associates each input value with an independent weight that canbe updated. The decision process of the CNN can be used forclassification. For example, the bottom layer of the CNN can takeinitial input data, such as an image or matrix, and the top layer of theCNN can output a label representing a decision made for the initialinput data by the CNN, such as a category of the image.

During a supervised training method for the CNN model, for example, aset of data (e.g., a training set) can be used as inputs to the CNNmodel for classification. Each input of the training set is assignedwith a pre-determined label (ground truth labels) for determiningwhether the classification of the CNN is correct. Based on thecomparison of the output labels and ground truth labels, the weights ofnodes can be updated for optimization. A CNN model trained by such aprocess can be used as a classifier for other input data of the sametype with the training set.

In some implementations, the CNN model can take the second feature dataassociated with the user (e.g., valid feature data after RPCA) and thedefined biometric data associated with the known individual (e.g.,template data) as input data, and provide an output label as a result ofdeciding whether a match exists between the second feature data and thedefined biometric data. For example, the output label can be text of“match” or “not match”, or any numerical or symbolic label, orindication that is assigned the meaning of “match” or “not match”. Foranother example, the output label can be a name, a profile, a username,a password, or any information that is associated with a user.

In some implementations, input data for the CNN model can be prepared invarious ways based on extracted periodic fragments. In someimplementations, a periodic fragment extracted from biometric dataassociated with the user (e.g., a signal fragment) can be combined withanother periodic fragment extracted from biometric data associated withthe known individual (e.g., a defined fragment). For example, N signalfragments and N defined fragments can be paired and stacked to form a2*N image, which can be used as input data for the CNN model to generatean output label.

In some implementations, the CNN model can use a difference matrix orimage as a sole input, in which the difference matrix or image isdetermined as the difference between the second feature data associatedwith the user and the defined biometric data associated with the knownindividual. For example, the second feature data can be the featurematrix associated with the user (e.g., a user feature matrix), and thedefined biometric data can be the feature matrix associated with theknown individual (e.g., a defined feature matrix). Accordingly, thedifference matrix can be a matrix that captures the difference ofcharacteristics between the two matrices. For example, the differencematrix can be a K*N matrix generated by subtracting a K*N definedfeature matrix from a K*N user feature matrix, or vice versa.

As an example, FIG. 7 shows a flowchart of a method for comparing an ECGsignal of a user with an ECG signal of a known individual according toimplementations of this disclosure. In this example, a difference matrixis determined based on an ECG signal of a user and an ECG signal of aknown individual. Process 700 can be used either in a setup process orin a practical scenario (e.g., testing process) such as real-timeidentification of a user. A trained CNN model can take on, as input, adifference matrix generated as a result of process 700, and output alabel (e.g., “match” or “not match”) for the real-time identification ofthe user in a test.

At operation 702, the ECG signal of the user is received.

At operation 704, the ECG signal of the known individual is received.

At operation 706, feature extraction operations, which can include, forexample operations 604-608, are performed separately for the two ECGsignals.

At operation 708, a feature matrix for the user is determined, namely auser feature matrix.

At operation 710, a feature matrix for the known individual is alsodetermined, namely a defined feature matrix.

At operation 712, a difference matrix is determined between the userfeature matrix and the defined feature matrix.

In some implementations, after the difference matrix is determined, theCNN model can decide whether a match exists or not between the userfeature matrix and the defined feature matrix based on the differencematrix, and provide a label representing the decision. If the outputlabel is “match” or equivalent, the user and the known individual isconsidered to be the same person; otherwise, if the output label is “notmatch” or equivalent, the user and the known individual is considered tobe different persons.

In some implementations, prior to testing (e.g., real-timeidentification), the CNN model can be trained in the setup process thatcan be illustrated as a flowchart in FIG. 8 according to implementationsof this disclosure. A set of difference matrices generated by process700 can be used as training data (e.g., a training set) for the CNNmodel. The difference matrices are determined between biometric dataassociated with the two individuals, in which the two individuals can beeither the same person or different persons. Each difference matrix cancome with a ground truth label, representing the relationship betweenthe two individuals (e.g., “match” or “not match”). Both the differencematrices and the ground truth labels can be used as training data forthe CNN model during the setup process.

At operation 802, a difference matrix determined as in operation 712 isinput to the CNN model. The weights of nodes of the CNN model areinitialized with values. A label is output by the CNN model.

At operation 804, the label output at operation 802 is compared with theground truth label that comes with the input difference matrix. If theoutput label is different from the ground truth label, theclassification is incorrect, and the weights of the CNN areautomatically updated and stored for use in next calculation. If theoutput label is the same as the ground truth label, the classificationis correct and the weights are kept unchanged.

At operation 806, optionally, operations 802-804 can be performed foreach difference matrix in the training set, repeated for at least onetime. This process can be repeated until the rate of correctclassification reaches a pre-determined level, such as, for example,90%. In some implementations, this process can be repeated until apre-determined running duration is reached without the rate of correctclassification reaching the pre-determined level. When the repeatedprocess ends, the CNN classifier completes the setup process and isready for practical use.

FIG. 9 shows a flowchart of a method for determining whether a matchexists using a difference matrix as an input for a CNN model accordingto implementation of this disclosure. Process 900 can be used either ina setup process or a real-time scenario for practical use.

At operation 902, a difference matrix is input to the CNN model. Thedifference matrix can be generated either in the setup process or duringreal-time identification of a user in contact with device 110.

At operation 904, a label of “match” or equivalent, or a label of “notmatch” or equivalent, is generated by the CNN model, which representsthe identification result.

Back to FIG. 6, at operation 612, a signal indicating that the singleindividual is the known individual is transmitted using a communicationchannel. The signal can be transmitted by communication device 340. Insome implementations, the transmitted signal can be used for furtherindication or identification. For example, the signal can be transmittedback to biometric identification device 110, which sends a visual,acoustic or haptic indication to the user. For another example, thesignal can be transmitted to the intermediate device, which sends avisual, acoustic or haptic indication to the user. For another example,the signal can be transmitted to a device not belonging to the user foridentification purpose, such as a gate control. For another example, thesignal can be transmitted to an application running at a computingdevice, which further uses the signal to finish identification andauthentication process for the user. However, implementations of thetransmission of the signal for identification purpose are not limited tothe examples given here.

In some implementations, biometric identification device 110 can beconfigured to have partial capability of executing the entire of process600, e.g., a wristband shown in FIG. 4. Such device can execute someoperation of process 600, e.g., operation 602, and have the rest ofprocess 600 executed by an external server, being either computingdevice 112 or an intermediate device. In such implementation, biometricsensor 330 can be a biometric sensor coupled to body 420 to produce abiometric measurement signal when activated by contact with a user.Communication device 340 can be one or more communication device coupledto body 420 controlled by CPU 210 to wirelessly transmit the biometricmeasurement signal to an external server, wherein first feature data isgenerated based on periodic fragments extracted from the biometricmeasurement signal, wirelessly receive a first signal from the externalserver indicative of a biometric identification data generated from thebiometric measurement signal, wherein the first signal is determinedbased on second feature data generated by removing data from the firstfeature data using RPCA, and the second feature data is compared withdefined biometric data associated with a known individual using an imageprocessing technique to determine whether a match exists indicating thatthe single individual is the known individual, and wirelessly transmit asecond signal to an identification device in response to a matchindicating that the user is the known individual.

In some implementations, communication device 340 can include a firstwireless communication device configured to wirelessly transmit abiometric measurement signal from biometric sensor 330 to the externalserver and wirelessly receive the first signal from the external serverindicative of a match exists indicating that the single individual isthe known individual, and a second wireless communication deviceconfigured to wirelessly transmit a second signal to the identificationdevice in response to the first signal.

In some implementations, biometric identification device 110 can beconfigured to have full capability of executing the entire of process600, e.g., a “smart” watch shown in FIG. 5. In such implementations, CPU210 is configured to execute instructions stored in memory 220 toreceive a biometric measurement signal generated by biometric sensor 330by contact with a user, extract periodic fragments from the biometricmeasurement signal, generate first feature data based on the extractedperiodic fragments, determine second feature data from the first featuredata by removing data from the first feature data using RPCA, determinewhether a match exists between the second feature data and definedbiometric data associated with a known individual by processing thesecond feature data and the defined biometric data using an imageprocessing technique, and in response to determining a match existsbetween the second feature data and the defined biometric data, transmita signal indicating that the single individual is the known individual.The image processing technique can be a DL technique that comprises aCNN model. The second feature data and the defined biometric data can beused as inputs to the CNN model to determine whether a match exists.

Optionally, in some implementations, the aforementioned biometricidentification device 110 can further have the capability of generatingthe defined biometric data in a setup process. In such implementations,CPU 210 is configured to execute instructions stored in memory 220 toreceive data based on periodic fragments associated with the knownindividual and extracted during the setup process, generate definedbiometric data (e.g., template data) from the data based on periodicfragments associated with the known individual and extracted during thesetup process using a same technique used to determine the secondfeature data, determine a valid feature set and an outlier feature setby performing RPCA on the first feature set and perform an orderingoperation on the valid feature set to determine the second feature setthat is determined using a top portion of the ordered valid feature set.The second feature set can be used as the defined biometric data herein.

Technical specialists skilled in the art should understand that, theimplementations in this disclosure may be implemented as methods,systems, or computer program products. Therefore, this disclosure may beimplemented in forms of a complete hardware implementation, a completesoftware implementation, and a combination of software and hardwareimplementation. Further, this disclosure may be embodied as a form ofone or more computer program products which are embodied as computerexecutable program codes in computer writable storage media (includingbut not limited to disk storage and optical storage).

This disclosure is described in accordance with the methods, devices(systems), and flowcharts and/or block diagrams of computer programproducts of the implementations, which should be comprehended as eachflow and/or block of the flowcharts and/or block diagrams implemented bycomputer program instructions, and the combinations of flows and/orblocks in the flowcharts and/or block diagrams. The computer programinstructions therein may be provided to generic computers,special-purpose computers, embedded computers or other processors ofprogrammable data processing devices to produce a machine, wherein theinstructions executed by the computers or the other processors ofprogrammable data processing devices produce an apparatus forimplementing the functions designated by one or more flows in theflowcharts and/or one or more blocks in the block diagrams.

The computer program instructions may be also stored in a computerreadable storage which is able to boot a computer or other programmabledata processing device to a specific work mode, wherein the instructionsstored in the computer readable storage produce a manufactured productcontaining the instruction devices which implements the functionsdesignated by one or more flows in the flowcharts and/or one or moreblocks in the block diagrams.

The computer program instructions may also be loaded to a computer oranother programmable data processing device to execute a series ofoperating procedures in the computer or the other programmable dataprocessing device to produce a process implemented by the computer,whereby the computer program instructions executed in the computer orthe other programmable data processing device provide the operatingprocedures for the functions designated by one or more flows in theflowcharts and/or one or more blocks in the block diagrams.

Apparently, the technical specialists skilled in the art may perform anyvariation and/or modification to this disclosure by the principles andwithin the scope of this disclosure. Therefore, if the variations andmodifications herein are within the scope of the claims and otherequivalent techniques herein, this disclosure intends to include thevariations and modifications thereof.

What is claimed is:
 1. A method for processing a biometric measurementsignal using a computing device, comprising: receiving a biometricmeasurement signal generated by contact with a single individual;extracting at least one periodic fragment from the biometric measurementsignal; generating, by the computing device and from the biometricmeasurement signal, first feature data at least partially based on theat least one extracted periodic fragment; determining, by the computingdevice, second feature data from the first feature data by removing datafrom the first feature data using robust principal component analysis;determining whether a match exists between the second feature data anddefined biometric data associated with a known individual by processingthe second feature data and the defined biometric data using a machinelearning technique; and in response to determining a match existsbetween the second feature data and the defined biometric data,transmitting, using a communication channel, a signal indicating thatthe single individual is the known individual.
 2. The method of claim 1,wherein the machine learning technique comprises a deep learning (DL)technique.
 3. The method of claim 2, wherein the DL technique comprisesa convolutional neural network (CNN) model, and the second feature dataand the defined biometric data are used as inputs to the CNN model todetermine whether a match exists.
 4. The method of claim 1, wherein thefirst feature data comprises an aggregated image, wherein the aggregatedimage is generated by aggregating at least two periodic fragmentsextracted from the biometric measurement signal.
 5. The method of claim3, wherein the at least one periodic fragment is represented by an imageor a vector, and aggregating at least two periodic fragments extractedfrom the biometric measurement signal comprises: stacking the imagerepresenting the at least one periodic fragment on top of another imageto generate the aggregated image in response to a determination that theat least one periodic fragment is represented by an image; and placingthe vector representing the at least one periodic fragment side by sidewith another vector to generate the aggregated image in response to adetermination that the at least one periodic fragment is represented bya vector.
 6. The method of claim 1, further comprising: receiving dataat least partially based on periodic fragments associated with the knownindividual and extracted during a setup process; and generating, usingthe data at least partially based on periodic fragments associated withthe known individual and extracted during a setup process, the definedbiometric data using a same technique used to determine the secondfeature data before determining whether a match exists between thesecond feature data and defined biometric data.
 7. The method of claim1, wherein the biometric measurement signal comprises anelectrocardiogram (ECG) signal.
 8. The method of claim 6, wherein the atleast one periodic fragment comprises a PQRST fragment.
 9. The method ofclaim 1, wherein removing data from the first feature data using robustprincipal component analysis comprises: removing outliers and noisesfrom the first feature data using robust principle component analysis.10. The method of claim 1, wherein determining, by the computing device,second feature data from the first feature data further comprises:determining valid data and outlier data from the first feature data byperforming robust principal component analysis on the first featuredata; and performing an ordering operation on the valid data todetermine the second feature data.
 11. The method of claim 10, whereinthe second feature data is determined as a top portion of the valid dataafter the ordering operation has been performed on the valid data. 12.The method of claim 1, wherein determining whether a match existsbetween the second feature data and defined biometric data associatedwith a known individual comprises: determining, at the computing device,a difference matrix based on the second feature data and the definedbiometric data associated with the known individual.
 13. An apparatusfor processing a biometric measurement signal, comprising: anon-transitory memory; and a processor configured to executeinstructions stored in the non-transitory memory to: receive a biometricmeasurement signal generated by contact with a single individual;extract at least one periodic fragment from the biometric measurementsignal; generate first feature data at least partially based on the atleast one extracted periodic fragment; determine second feature datafrom the first feature data by removing data from the first feature datausing robust principal component analysis; determine whether a matchexists between the second feature data and defined biometric dataassociated with a known individual by processing the second feature dataand the defined biometric data using a machine learning technique; andin response to determining a match exists between the second featuredata and the defined biometric data, transmit, using a communicationdevice, a signal indicating that the single individual is the knownindividual.
 14. The apparatus of claim 13, further comprising: a body; abiometric sensor coupled to the body to produce the biometricmeasurement signal when activated by contact with the single individual;and the communication device coupled to the body and controlled by theprocessor to transmit the signal, to a reader device, indicating thatthe single individual is the known individual.
 15. The apparatus ofclaim 13, wherein the instructions further comprise instructions to:receive data at least partially based on periodic fragments associatedwith the known individual and extracted during a setup process;generate, using the data at least partially based on periodic fragmentsassociated with the known individual and extracted during a setupprocess, the defined biometric data using a same technique used todetermine the second feature data before determining whether a matchexists between the second feature data and defined biometric data;determine, based on the first feature set, a valid feature set and anoutlier feature set by performing robust principal component analysis onthe first feature set; and perform an ordering operation on the validfeature set to determine the second feature set, wherein the secondfeature set is determined using a top portion of the ordered validfeature set.
 16. The apparatus of claim 13, wherein the biometricmeasurement signal comprises an electrocardiogram (ECG) signal.
 17. Theapparatus of claim 16, wherein at least one periodic fragment comprisesa PQRST fragment.
 18. The apparatus of claim 13, wherein the machinelearning technique comprises a deep learning (DL) technique, the DLtechnique comprises a convolutional neural network (CNN) model, and thesecond feature data and the defined biometric data are used as inputs tothe CNN model to determine whether a match exists.
 19. An apparatus,comprising: a body; a biometric sensor coupled to the body to produce abiometric measurement signal when activated by contact with an singleindividual; a memory; and at least one communication device coupled tothe body and controlled by a processor to: wirelessly transmit thebiometric measurement signal from the biometric sensor to an externalserver, wherein first feature data is generated at least partially basedon at least one periodic fragment extracted from the biometricmeasurement signal; wirelessly receive a first signal from the externalserver indicative of a biometric identification data generated from thebiometric measurement signal, wherein the first signal is determinedbased on second feature data generated by removing data from the firstfeature data using robust principal component analysis, and the secondfeature data is compared with defined biometric data associated with aknown individual using a machine learning technique to determine whethera match exists indicating that the single individual is the knownindividual; and wirelessly transmit a second signal to an identificationdevice in response to a match indicating that the single individual isthe known individual.
 20. The apparatus of claim 19, wherein the atleast one communication device comprises: a first wireless communicationdevice configured to wirelessly transmit a biometric measurement signalfrom the biometric sensor to the external server and wirelessly receivethe first signal from the external server indicative of a match existsindicating that the single individual is the known individual; and asecond wireless communication device configured to wirelessly transmit asecond signal to the identification device in response to the firstsignal.